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Abstract 

Our initial intention was to set out to explore the potential for implementing outcome-based assessment 
practices in a first year statistics unit. It has evolved into a search for a pedagogy enabling clear definition of the 
desired learning outcomes in our students, and alignment of instruction, student resources and assessment 
with these outcomes. Previous learning and assessment materials were used as starting points for the pilot 
study. The aim was to develop a model which might feasibly be repeated in the context of our subject and 
related ones, and which might be potentially applied in other disciplines. A recent revision of Blooms 
Taxonomy was chosen to facilitate classification of an intended hierarchy of desired knowledge, processes and 
skills. Unit objectives were behaviourally framed to provide identifiable indicators of student outcome 
achievement. Assessment was similarly classified to align the desired and observable outcomes. Appropriate 
instructional techniques and authentic assessment were chosen commensurate with student interest and the 
development of higher order cognitive skills. An evaluation rubric was used to clarify the requirements of the 
assessment task. This provided focus material to foster organization of responses. To prevent students 
privileging one outcome above another, no marking schema was included. Students worked independently in 
teams of two to complete complementary sections of the assignment. However discussion of their work with 
their partner was not only expected, but encouraged. Team solution fostered cooperative learning without 
providing opportunities for plagiarism. Reflective practices and student and staff feedback throughout the 
study led to refinements in the procedures. Summative evaluation encompassed results of student assessment 
and surveys and interviews of both staff and students. Semantic problems resulted in difficulties in using the 
revised Blooms taxonomy to classify assessment questions. The word explain at times required 
demonstration with a simple diagram, at others, identifying the elements of a formula or even analysis to 
determine the relevance of a particular model. There is an obvious potential for variability in interpretation of 
the skills required by some questions and hence the possibility of a lack of reliability in classification of the 
objectives and assessment. A need arises therefore to refine the taxonomy for more specific use within the 
discipline. 


This journal article is available in Journal of University Teaching & Learning Practice: http://ro.uow.edu.au/jutlp/voll/iss2/5 
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Abstract 

Our initial intention was to set out to explore the potential for implementing 
outcome-based assessment practices in a first year statistics unit. It has evolved 
into a search for a pedagogy enabling clear definition of the desired learning 
outcomes in our students, and alignment of instruction, student resources and 
assessment with these outcomes. Previous learning and assessment materials 
were used as starting points for the pilot study. The aim was to develop a model 
which might feasibly be repeated in the context of our subject and related ones, 
and which might be potentially applied in other disciplines. 

A recent revision of Bloom’s Taxonomy was chosen to facilitate classification of 
an intended hierarchy of desired knowledge, processes and skills. Unit objectives 
were behaviourally framed to provide identifiable indicators of student outcome 
achievement. Assessment was similarly classified to align the desired and 
observable outcomes. Appropriate instructional techniques and authentic 
assessment were chosen commensurate with student interest and the 
development of higher order cognitive skills. 

An evaluation rubric was used to clarify the requirements of the assessment 
task. This provided focus material to foster organization of responses. To 
prevent students privileging one outcome above another, no marking schema 
was included. Students worked independently in teams of two to complete 
complementary sections of the assignment. However discussion of their work 
with their partner was not only expected, but encouraged. Team solution fostered 
cooperative learning without providing opportunities for plagiarism. 

Reflective practices and student and staff feedback throughout the study led to 
refinements in the procedures. Summative evaluation encompassed results of 
student assessment and surveys and interviews of both staff and students. 

Semantic problems resulted in difficulties in using the revised Bloom’s 
taxonomy to classify assessment questions. The word ‘explain’ at times required 
demonstration with a simple diagram, at others, identifying the elements of a 
formula or even analysis to determine the relevance of a particular model. There 
is an obvious potential for variability in interpretation of the skills required by 
some questions and hence the possibility of a lack of reliability in classification of 
the objectives and assessment. A need arises therefore to refine the taxonomy 
for more specific use within the discipline. 
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Introduction 

In any instructional process, one of the fundamental questions that is continually asked by 
the teacher remains: “But how will I know when they understand the concepts.” This question 
has long been posed and has formed the crux of the dilemma facing practitioners today. But 
if this is a problem for the educators themselves, it is mirrored in the minds and hearts of 
students the world over. How do they know just what are the knowledge and skills required of 
them and how do they show that they have acquired them? 

This study was born of the teachers’ earnest attempts to provide focused assessment that 
did indeed assess what it claimed to assess, and also provided ‘signposts’ for the learning of 
her students. Such focused assessment could afford a guiding framework for teaching and a 
powerful learning tool for students. 

There have been recent large-scale changes in syllabi in the state high school system of 
N.S.W. and in the United States founded on outcome-based learning and assessment, but 
relatively little evidence of this approach exists in our universities. Many courses are content 
based, with ‘lip service’ paid to outcomes only evident in a broad specification within unit 
outlines. 

To design a framework for guiding both teaching and learning has required consideration of: 

• the processes of learning; 

• learning styles/needs; 

• the definition of learning; 

• appropriate forms of assessment. 

The framework needed to be developed in the context of the globally defined graduate 
outcomes and through defined course content. 

As the problem that arises in the classroom, the classroom should provide an appropriate 
venue for its resolution. Action research affords an appropriate methodology for this type of 
study. (Gomm, 2004) The participant researchers’ collective experiences, observations, 
reflections and analyses of practice in previous implementations of the research cycle were 
used to identify the successful strategies and student needs. These form the starting point for 
this study of a first year statistics program and its assessment. Following the development of 
an appropriate model and its subsequent implementation: 

• staff and student evaluations; 

• review of students’ assessment performance; 

• reflective practices 

have led to modifications, eliminations and further inclusions as the framework evolves. 

Theoretical background 

Learning theories 

The processes of student learning require due attention in designing assessment that aligns 
with all aspects of teaching and learning. The three main theories of learning are: 

• behaviourist; 

• cognitive; 

• constructivist. 
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The construction of the learning framework has been supported by what the researcher’s 
have perceived as compatible with the three main learning theories. However, the primary 
researcher’s social constructivist perspective may still possibly be discerned. Decisions were 
made after surveying the types of strategies used by devotees of each of the three theories 
and still others not reported in this paper. Anderson and Elloumi et al (2004) provide a 
structured outline of the required theoretical background. 

Behaviourism perceives learning as resulting in changes in behaviour. Some behaviourist 
considerations which could be implemented could include: 

• clear definition of expected outcomes to provide focus; 

• sequencing conducive to learning; 

• opportunities for self testing of achievements of these outcomes; 

• interactive feedback on achievement. 

The cognitive approach regards learning from the ‘information processing’ viewpoint and 
includes the aspects of motivation, memory, thinking and reflection. Practitioners see 
learning as taking place when knowledge is internally accommodated and personalized. 
From this stand, teaching should involve: 

• checks to identify presence of appropriate and existing cognitive structures, otherwise 
provision of ‘advance organizers’ where there are none (Ivie 1998); 

• ‘chunking’ of material to be learned; 

• visual formatting of materials and careful sequencing of content to catch attention and 
promote clear perception of the desired learning; 

• provision of connective links between higher and lower levels of learning; 

• use of models; 

• setting of expectations; 

• opportunities to refine learning through application, analysis, synthesis and evaluation 
to contextualize the learning. (Blanton, 1998) 

Constructivists view the teacher as the facilitator in the learning process. They believe that 
learning takes place as the student actively participates, interpreting, processing and 
constructing new knowledge. Because they see learning as contextual they consider it to be 
more effective in practical situations. 

In designing the pedagogy, consideration should be given to: 

• active learning processes; 

• interactive learning which allows collaboration of instructor and students in the 
process; 

• a cooperative learning environment; 

• tasks which provide individual engagement of the learner; 

• opportunities for reflection; 

• meaningful learning experiences which relate to the student’s own ‘world’, either as a 
student of their selected discipline or their ‘real life’. 
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Motivation is fundamental to successful learning. It can be either intrinsic (derives from within 
the learner e.g. self satisfaction/sense of achievement) or extrinsic (usually generated by the 
instructor or course e.g. marks/grades, praise etc). Intrinsic motivation appears to be more 
powerful in eliciting deeper and life-long learning. (Avery 1999) Song and Keller (1999) 
proposed a model for motivating students that endeavoured to: 

• catch attention; 

• offer relevance; 

• develop confidence (by defining expectation); 

• promote satisfaction (through interactive feedback). 

Any approach to learning needs to include strategies that will develop or enhance meta- 
cognitive skills. Although most intrinsically motivated students display these skills to some 
degree, any attempt to place students in charge of their own learning, needs to promote 
them. 

Learning styles 

If the assessment is to be ‘fair’ to all students then it should offer appeal to all styles of 
learner. Kolb’s experiential learning theory describes learning as taking place across two 
intersecting continua of perception and processing. He observed that learners appeared to 
demonstrate preferences for pairs of the phases of learning. These pairs of preferences lie at 
either end of either of the two continua. 


Figure 1. Phases of Kolt’s Learning Cycle. (Nieweg, 2000) 




Reflective Concrete 

Observation X Experience 


Abstract Active 

Conceptualisaion Experimentation 
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Kolb described learners as: 

• divergers who prefer concrete experience and reflection on that experience; 

• assimilators who prefer reflection on and conceptualization of the experience; 

• convergers who conceptualise the experience and then experiment actively with the 
idea; 

• accomodators who prefer concrete experience and the opportunity to experiment with 
ideas formed by the experience. (Kolb 1984) 

However, the most enriching learning experiences afford the opportunity to all participants to 
work cyclically through all four phases. Tasks that expose students to all of these phases are 
more likely to stimulate deeper learning. (Kolb 1984; Nieweg 2000) 

Since interactive learning also supports deeper learning, tasks that allow collaboration, but 
still encourage independence and ingenuity need to be developed. The ‘team’ approach to 
assessment tasks also promotes confidence and pays more than ‘lip service’ to espoused 
graduate attributes that applaud ‘community’ values and ‘sharing of talents’. (University of 
Wollongong, 2004) Tasks should also seek to use available technology as the community 
increasingly demands these skills of graduates and provide ‘authentic’ (Cobb, 1993; Avery, 
1999; Evans, 1999; Prestidge and Glaser, 2000) or meaningful tasks which are modelled on 
those previously experienced. 

Defining the learning 

Even defining such a supportive framework for learning, both teachers and students need to 
be aware of the knowledge and skills the students are expected to demonstrate, i.e. the 
expected learning should be clearly defined. It is such definition that has proved expeditious 
in student learning in the high school system. The aim here has been to explicitly align the 
expected learning with graduate attributes, course and unit outcomes, teaching objectives 
and pedagogy in clear statements of the objectives available to teacher and student alike. 
This alignment process focuses the subject delivery directly at student learning and it has 
become increasingly evident to the participant researchers that this process is fundamental 
to successful teaching/learning. 

Although we inform students of what they are expected to know, many are at a loss as to 
how they are to show such learning. The assessment tasks may be modelled on previous 
class work, but ‘marking guides’ may clarify what is expected and offer ‘organizers’ for 
responses. (Ivie 1998; Heady 2000; Prestidge and Glaser 2000; Romagnano 2001; 
Montgomery 2002) The ‘guides’ may be explicit in initial tasks, but become less so as 
student learning progresses. Careful thought needs to be given to inclusion of marks on 
these ‘guidelines’, as explicit and narrow alignment of marks with tasks may detract from the 
task as a learning tool. (Black, 1998) The ‘guides’ were given to students without the marks 
in order to offer structure for responses. This was because the premise held by the 
researchers that their inclusion would: 

• reinforce the extrinsic motivational value of aggregating marks at the expense of the 
intrinsic value of accumulating knowledge; 

• encourage the ‘pass/fail’ mentality; 

• detract the value of ‘organizers’ as a ‘scaffold’ for giving ‘complete’ responses. In turn 
this may diminish student confidence in their capacity to provide complete answers. 
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• remove focus from the ‘guideline’ as an ‘advance organizer’. Such tools afford 
structures which work to enhance a student’s capacity to organize their own learning 
(meta-cognitive skills). For example, comparing two data sets may require observation 
and analysis of a number of different features of each set. The detail may be clearly 
defined in the ‘guide’ for the first assessment task, explicitly defined or even omitted in 
subsequent tasks. 

• encourage students to allocate importance of the knowledge based on the relative 
weighting of the marks allocated. The intention, however, should be to encourage 
students to regard all aspects of the task as important and worthy of doing well. 

Fundamental to the design of a working framework that recognizes this theoretical 
background, is the statement of expected learning, i.e. the statement of the assessment 
objectives. Evaluation of student learning can be very subjective without some generalized 
frame of reference. Bloom and his associates pursued early research in this field. Their 
original intention was to devise a taxonomy of learning which would facilitate discussion by 
educators. Bloom’s taxonomy offered a classification of knowledge and skills in what 
purported to be a hierarchical structure, highlighting representation of both higher and lower 
order knowledge and skills. Thus it became a useful tool for evaluation of assessment 
against teaching objectives. Framing the objectives in behavioural terms enabled observation 
of their achievement. (Bloom 1974) 

Such behavioural specification earned the taxonomy a great deal of criticism. Anderson, 
Krathwohl et al (2001) have highlighted some of the criticisms leveled at Bloom. They refer to 
claims that it was only useful in the realm of the behaviourists, recognizing learning as only in 
an altered pattern of behaviour. However this criticism fails to see the objective as a means 
towards recognizing learning rather than as the end in itself. They also discuss the criticism 
that it ‘limited’ recognition of achievement. Specificity in defining an objective may simplify 
assessment of its achievement, but may also cloud recognition of any more complex learning 
being exhibited. It can prove a challenging exercise to define creative achievement in the 
taxonomy’s terms and its application may be less than reliable for different educators from 
different fields. (Anderson, Krathwohl et al. 2001) 

Anderson, Krathworl et al (2001) undertook a revision of Bloom’s taxonomy, separating 
knowledge from the cognitive processing dimension to form a two-dimensional classification 
of knowledge and skills. They recognized the inherent lack of hierarchy in the taxonomy, but 
maintained that an evident hierarchy did exist within each classification. (Anderson, 
Krathwohl et al. 2001) 
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Table 1: Two-Dimensional Cross-Classification of Types of Knowledge 
by Cognitive Processing Skill 


Knowledge 

Dimension 

Remember 

Cognitive Processes Dimension 

Understand Apply Analyse Evaluate 

Create 

Factual 







Conceptual 







Procedural 







Meta-cognitive 







(Anderson, Krathwoh 

1 et al. 2001) 


Although the revision of Bloom’s taxonomy was chosen for this study, it is not the only 
classification tool available to educators. Another, the SOLO taxonomy, is closely aligned 
with the developmental stages of Piagetian theory. The taxonomy classifies learning into 
levels of increasing complexity. Learning at the lowest stage (pre-structural) does not 
recognize connections between pieces of information. Whereas at the highest level 
(extended abstract), connections are made not only in the current context but extend beyond. 
(Biggs and Collis 1982) 


Table 2: The phases of learning as described in the SOLO taxonomy 


Demonstrated learning 

Pre structural 

Unistructural 

Multistructural 

Relational level 

Extended abstract 
level 

Students acquire disconnected information 






Students make simple obvious connections 






Students make many connections 






Students demonstrate the relationship 
between connections 






Students demonstrate overview of the 
connections 






Students extend connections beyond the 
current context 






Students generalize and transfer from specific 
to abstract 







(Biggs and Collis 1982) 


Journal of University Teaching and Learning Practice 
































Assessment is bloomin' luverly: Developing assessment that enhances learning 
Maureen Morris, Anne Porter and David Griffiths 


The learning framework 

In the light of the surveyed theory, the learning framework has been chosen to include: 

• content which has been appropriately ‘chunked’ and sequenced; 

• experiential and interactive learning opportunities; 

• collaborative tasks and assessments; 

• relevant and engaging tasks and assessments; 

• marking guides as ‘advance organizers’; 

• marking criteria and ‘worked’ solutions to provide detailed and timely feedback. 

All facets of the subject’s presentation and evaluation were aligned through the objectives. It 
is this process of alignment which appears to have underpinned any success which has been 
observed in implementing the selected framework. 

The targeted level of student learning was evaluated by ‘coding’ question of assessment 
tasks using the revised Boom’s taxonomy, although this proved a less than reliable task. 


Figure 2: Teaching learning framework showing selected learning strategy inputs 
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Implementation 

In the subject that was the focus of this study, assessment was comprised of: 

• three assignments; 

• midterm test (not treated in detail in this paper); 

• Laboratory tasks which were completed and recorded in a Lab manual 

• Oral Presentation (not treated in detail in this paper); 

• Final exam. 

All were aligned with the objectives and with each other. The focus for this paper is the 
assignments. However alignment of all key components means that evaluation of the 
assignments involves due consideration to all facets of the subject which impact upon the 
teaching and learning. 

The assignments 

Statements of the specific objectives accompanied each assignment together with a set of 
marking guides (no marks indicated) that provided students with indicators of desired 
outcomes and hence structures for responses. As students were presumed to have learned 
from previous assignments, the Marking Guidelines of the second and third assignments 
were less detailed. Students were required to use a statistical software package for data 
analysis and to word-process assignments, interweaving output with their own words. 

Assignments were completed in teams of two. Each assignment question was comprised of 
two parts and each member responded to one only. These parts were designed to 
complement or supplement each other, and team members were encouraged to collaborate 
and share responses, but present their own solution to their own part of the problem. They 
could not ‘collaborate’ with anyone other than their partner. For example, in Assignment 2, 
one student would complete questions dealing with a model from a Poisson distribution, 
generating expected frequencies for a sample based on the given model (working with 
theory). The partner would generate a random sample and answer questions based upon 
that sample and, using expected frequencies from their partner’s output, would conduct a 
‘Goodness of Fit’ Test for their sample (working with their simulated data). In the next 
question, the roles were reversed for a question dealing with the Binomial distribution. This 
approach ensured the need for collaboration. Student Evaluation responses indicate that the 
‘teamwork’ appears to have promoted confidence in the students. It afforded opportunity for 
discussion and clarification of more complex tasks without having to surrender independence 
of response. 

The questions were aligned with appropriate laboratory tasks in order to reinforce prior 
learning. In an attempt to engage students in analysis of ‘real’ data, the tasks required 
access to data files generated from data collected through a web-based survey of the 
students themselves. The last assignment involved the student’s personal collection of data 
in addition to its analysis. 

Attempting to exercise some measure of control over student assessment, tutors were 
cautioned about responding to questions relating to the assignments. They were instructed to 
direct students to appropriate lecture and laboratory material and not to model ‘answers’ with 
similar or related problems. This was to reduce the chance of diminishing the ‘quality’ of 
anticipated learning. Guided and repetitious performance demands far less of students than 
when minimal guidance is given. 
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The marking criteria (not provided in advance to students) listed a set of targeted responses 
each of which was marked on a 0/1 scale. This facilitated prompt and detailed feedback that 
simply listed what had been achieved (one mark) and what had not (zero marks). Students 
have perceived the whole assessment process as ‘pretty fair’, since generally each student 
could potentially earn ‘what they deserve’ and be neither limited nor completely supported by 
the other member of the team. Weaker team members received some help in terms of 
collaborative discussion, but could not end up with their submission completed by another. 
There appeared to be little evidence of illegitimate ‘collaboration’ (branded as ‘plagiarism’ in 
the assignment preamble) outside the team structure. The teachers’ perception is that such 
plagiarism appears to have been more prominent in previous approaches to assignments. 

The laboratory classes 

These classes provided collaborative and experiential learning exercises seen as imperative 
to learning and were conducted in computer rooms. The Lab manuals/workbooks contained 
the bulk of required learning materials, a course introduction, assignment specifications, 
objectives and tasks for each week, technical notes and PowerPoint slides of lecture 
material. Space was provided for pen-and-paper responses and to allow for students to 
paste in computer generated output. Thus students ultimately produced their own expanded 
workbook intended as a further valuable learning resource. Weekly tasks involved 
‘preparation’ questions, data collection (both qualitative and quantitative), organization, 
analysis and interpretation of output. 

Whilst a tutor provided support and guidance, the tasks were designed to encourage student- 
centred learning. The team approach was actively encouraged, particularly in the initial 
classes. This ultimately led to a relaxed and collaborative learning environment. 

Tasks draw on different theoretical ideas such as loosening and tightening constructs to 
engender creative thinking and then to consolidate ideas (Kelly, 1955). Still other tasks 
involve experiential learning (Kolb 1984; Nieweg 2000), while some emphasise collaborative 
learning (White, 1995), activity based exercises using real data and authentic assessment 
(Cobb, 1993; Avery, 1999; Evans, 1999; Prestidge and Glaser, 2000), and some promoted 
the development of metacognitive skills through reflection on activities and frames and 
concept mapping (Turban, 1990; Novak and Gowin, 1984). The sequencing of tasks typically 
follow a listening, speaking, reading, and writing pattern. 

To promote continued learning outside class hours, the tasks were designed to be too long to 
be completed in class time. However, students worked frenetically to complete each weekly 
task in class, resulting in a distorted perception of the workload. In future implementations 
this needs to be clarified with students to minimize stress. Marks were allocated (10%) for 
completion to maximum of 10 out of 12 labs. 

Worked solutions were later uploaded onto the Web. Because many students fell behind, 
they were encouraged to download solutions to complete the manual as they would be able 
to take this resource into the final exam. Most students realized eventually that output was of 
little use to them without some experience in its interpreting. The classes were busy but 
relaxed, with most students actively engaged at all times. Few students left the classes early, 
and although attendance was not compulsory, it remained stable, with most students 
appearing to recognize the relevance to the subject learning framework. 
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The midterm test 

The test was based on material from the first three quarters of the course and provided an 
opportunity for students to approach an exam armed with their self-created and (hopefully) 
completed Lab manual as a guide to expected learning. Familiarity with the material and their 
previous classroom experience appeared to both relax and encourage optimism in 
appropriately prepared students. 

The final exam 

The final exam was aligned with laboratory tasks and assignments. Students were permitted 
to take their completed laboratory manual into the exam as a reference. Because of the 
analytical approach taken in all facets of continuous assessment, questions represented a 
more interpretive and analytical approach than previously used in examinations in this 
subject. It was anticipated that students would find it more difficult than in previous years 
because of the increase in the higher order cognitive component. 

However students did appear more ‘comfortable’ with the subject and what was expected of 
them. Student Evaluations indicated lower stress levels ahead of the examination than had 
been evident in previous semesters. 

Evaluation 

Process 

Student learning was evaluated through the summative measures of marks (assignments, 
lab marks, midterm, presentation and final exam marks). Students were also surveyed as to 
their attitudes to the components of the subject’s delivery and the ranked level of importance 
to their learning. They chose from a set of specified scaled responses. Similarly they were 
also asked to select statements that reflected their level of learning in this subject. 

Qualitative data was also collected. Students nominated topics with which they felt 
‘comfortable’ and those they found ‘difficult’ (open text) and described their attitudes to the 
‘fairness’ of the assessment (open text). There was provision for further comment if they 
wished and some students volunteered to be interviewed personally. Triangulation of the 
data from all of these sources, together with attendance rates was to be used to support the 
belief that the framework had indeed enhanced student learning. (Bloor, 1997) Teaching 
staff were similarly surveyed. 

Results: student attitudes 

One hundred responded (from a total of 200 students enrolled). The following chart 
summarises students’ responses to which aspects of the learning environment were 
regarded as extremely important or moderately important to their learning. The table lists the 
responses ordered by proportion of respondents. 
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Table 3: Student responses (n=100) to perception of importance 
to learning (ranked by proportion) 


Component 

‘Extremely 

important’ 

‘Moderately 

important’ 

Total regarded 
as important 

Lab manual 

64% 

29% 

93% 

Lab task solutions on Web 

61% 

30% 

91% 

Assignments 

59% 

35% 

94% 

Lab classes 

54% 

34% 

88% 

Lecture Notes 

46% 

41% 

87% 

Lab Tasks 

39% 

49% 

88% 

Theory review in lab classes 

35% 

37% 

72% 

Marking guides (assignments) 

31% 

35% 

66% 

Midterm exam 

29% 

46% 

75% 

Lectures 

29% 

45% 

74% 

Forum 

13% 

39% 

52% 

Objectives in Manual 

7% 

37% 

44% 

Textbook 

7% 

19% 

26% 


* Students could not differentiate between these items and hence results are difficult to interpret. 


Students have indicated the importance of the Lab manual as a focus for completion of all 
assessment. It is also the only resource that they may take into the final exam, so they have 
worked diligently to complete it. The results indicate the value students have placed on active 
and collaborative learning experiences. They have perceived the assignments as strongly 
influencing their learning, but show their need for the detailed feedback to be found in worked 
solutions. They still appreciate the small group ‘teaching’ ( Theory review) of the tutor in lab 
classes, but also view the lecture notes as guides to essential content. Marking guides do not 
rate as highly as had been expected, but this may have been because it was an innovation 
and students appear to rarely read the entire preamble in assignments. Most students, 
however, felt that they had improved their achievement in assignments by using the Marking 
guides and hence they were considered as moderately to extremely important by 66% of 
respondents. 

Although many students claimed not to have used the Objectives (29%), interviews of both 
staff and students indicate that they were aware of their presence and purpose (despite 
claiming not to have referred to them). Perhaps their effect is operating subliminally as an 
‘organizer’ for the learning/teaching processes. Since the Marking guides and the objectives 
have also been defined in the Lab Manual and in each of the Assignments, the measure of 
their effect may be masked, and revealed only in further interview. 

Most students also appeared to have appreciated the Team” approach. Nearly 60% of 
students found it productive and useful. Expanded comments also lend support to the notion 
that it may have helped to reduce ‘cheating’, and increase confidence in the students. 
However, 13% of students perceived that the load had fallen upon one student (presumably 
themselves) in the two person team. 
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Table 4: Reported attendance rates (13 week semester) 


Subject 

10-13 weeks 

7-9 weeks 

Lectures (Tuesday- 2 hours am) 

58% 

24% 

Lectures (Thursday-1 hours) 

43% 

20% 


Attendance rates were reasonable as attendance is not compulsory. The subject has been 
designed as student-centred and has sufficient support resources in place to enable students 
to complete it independently. 


Results: student learning 

Table 5: Mean assignment and final exam scores (n=186) 


Task (relative weighting) 

Minimum (%) 

Maximum (%) 

Mean (%) 

Std. Deviation (%) 

Assignment 1 (10%) 

25 

100 

63 

16.9 

Assignment 2 (10%) 

16 

100 

69 

20.1 

Assignment 3 (10%) 

1 

100 

68 

20.3 

Midterm (5%) 

0 

100 

54 

20.0 

Exam Mark (50%) 

7 

100 

54 

18.4 


There is evidence of an increase in level of achievement between the first and second 
assignments, but this was not repeated between the second and third. There was some 
commonality between the first two tasks, but the third was vastly different and conceptually 
more difficult. It is heartening that there is little evidence of a decline in achievement! The 
midterm was held after the second assignment, and hence students benefited from learning 
in the prior tasks. The midterm was similarly structured to the final exam, and hence differed 
in format from the assignments. By the second and third assignments, some students were 
feeling the pressure of the workload, and hence either submitted trivial attempts or failed to 
submit at all. 


Table 6: Individual correlations (all significant at the 0.01 level of significance) 


Task 

Assignment 1 

Assignment 2 

Assignment 3 

Midterm 

Final Exam 

Assignment 1 

i 

0.39 

0.30 

0.40 

0.39 

Assignment 2 


1 

0.57 

0.29 

0.39 

Assignment 3 



1 

0.26 

0.43 

Midterm 




1 

0.73 

Final Exam 





1 
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Correlations between the assignments and the final exam are all weak, but positive and 
statistically significant (_=0.01). The significant cross correlations impact upon the any 
regression analysis involving all of these variables. The midterm is closer in style to the final 
exam, and benefits from the prior learning afforded particularly by assignment 1. 

The final exam mark (%) was regressed against the marks for the above listed assessment 
tasks (%).The following table gives abbreviated output. 

Table 7: The coefficients 1 


Model 

Coefficient 

Std Error 

t 

Sig 

(Constant) 

1.63 

4.06 

0.4 

0.70 

Assignment 1 

0.04 

0.06 

0.8 

0.45 

Assignment 2 

0.06 

0.05 

1.2 

0.25 

Assignment 3 

0.19 

0.05 

3.7 

0.00 

Midterm 

0.59 

0.05 

12.3 

0.00 


1 Predictors: (Constant), Midterm (%), Assignments (%), Assignment 1 (%), Assignment 2 (%) Dependent Variable: Exam (%) 


While 53% of the variation in exam marks is explained by the midterm, only 60% was 
explained by all of the variables. The coefficients are all positive, indicating that their impact 
is to increase achievement in the final exam, but only the coefficients of assignment 3 and 
the midterm are statistically significant. 

Although the regression analysis does not provide strong evidence of the value of the 
assignments in enhancing learning, it does afford supporting evidence which supplements 
the data from the other sources. 

Comparison of final grades between 2003 and 2004 in this subject does not indicate any 
obvious improvement for this implementation (a little disappointing for the researchers). 
However two subtle changes can be detected: 

1. There has been a slight reduction in the failure rate (but also a slight reduction in the 
percentage allocation of High Distinctions). Thus there has been a narrowing in the 
variance of student achievement levels. 

2. Coding of the exam questions using the revised Bloom’s taxonomy for this 
implementation and the previous one, indicate a shift in the level of learning to a 
higher order. Some basic knowledge and procedural application questions have been 
replaced by more analytic and interpretive ones. This would seem to indicate a ‘more 
difficult’ exam, but with no accompanying decrease in level of performance. (There 
has been no scaling of results for this subject). Maybe we have some evidence of 
success! 

Descriptions included in the work of Anderson, Krathwohl et al, again used to help classify 
the behaviours sought in assessment questions in the past two final exams. (Anderson, 
Krathwohl et al., 2001) This process has highlighted a shift in weighting from questions 
related to Procedural Application to those requiring Analysis and Evaluation - a shift to 
higher learning. 
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Results: administration 

The Marking Criteria have expedited marking this semester. All assignments were marked by 
one person and returned to students within one week. The process of right/ wrong marking 
has facilitated this. This approach needs to be carried through to the next implementation. 

Results: identifying the next step 

The task now is to transport the benefits of this experience into the next implementation. The 
aims of the researchers are now to: 

• Characterise the parts of the Lab manual which best illustrate how to set and 
sequence tasks to maximize learning, that is to identify ‘what works’ and ‘what doesn’t 
work’. The manual has been extremely successful. Implicit strategies need to be 
made more explicit in order to generalize and to assist others in implementation. 

• Refine the evaluation tools by eliciting attitudes to learning for the topics specified to 
the text responses in this implementation. There may even be an opportunity to 
explore student perception of learning in even greater detail. 

Conclusion 

Alignment has been fundamental to the processes that have been developed. It has been 
achieved through the use of the revised taxonomy of Bloom. However the classification of 
learning still remains both frustrating and time consuming. Simple requests such as ‘explain’ 
contain so many subtle nuances dependent upon the context. A student may be asked to 
‘explain using the output given’. This requires discriminating between relevant and irrelevant 
information and using it to justify. Alternatively, the students may be asked to ‘explain’ a 
concept with the aid of a diagram ( interpret ), or to ‘explain’ a process i.e. list the steps 
involved ( recall a procedure). These all demand different levels of skills. It would appear that 
without discipline specific vocabulary, such a general taxonomy is bound to prove difficult to 
use and use reliably! 

So the obvious question again repeats itself: “Is it possible to track assessment of knowledge 
and cognitive processing skills more effectively?” Most teachers have developed notions of 
levels of learning. Is it possible to formalize them to enable the tracking? Fair assessment 
would require definition of observable behaviours related to these notions. One potential 
solution might be provided through reflective observation of students learning. Some 
educators might be motivated to classify student learning using the following hierarchy: 

• What (Bloom’s Knowledge component?): Symbols/terms, formulae, procedures, 
concepts, theories or models; 

• How (Bloom’s Procedural component?): Simple/complex; 

• Why (Bloom’s Understand, Analyze, Evaluate, Create?)', 

• When (Transfer of knowledge and skills); 

• Personal Awareness (Bloom’s metacognitive component?). 

To effectively test the usefulness of this taxonomy, indicative behaviours need to be 
described for each of its categories. This too will form an integral part of further 
implementations in this continuing research into effective processes for teaching and learning 
in higher education. 
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